An algebra of human concept learning

An important element of learning from examples is the extraction of patterns and regularities from data. This paper investigates the structure of patterns in data defined over discrete features, i.e. features with two or more qualitatively distinct values. Any such pattern can be algebraically decomposed into a spectrum of component patterns, each of which is a simpler or more atomic ‘‘regularity.’’ Each component regularity involves a certain number of features, referred to as its degree. Regularities of lower degree represent simpler or more coarse patterns in the original pattern, while regularities of higher degree represent finer or more idiosyncratic patterns. The full spectral breakdown of a pattern into component regularities of minimal degree, referred to as its power series, expresses the original pattern in terms of the regular rules or patterns it obeys, amounting to a kind of ‘‘theory’’ of the pattern. The number of regularities at various degrees necessary to represent the pattern is tabulated in its power spectrum, which expresses how much of a pattern’s structure can be explained by regularities of various levels of complexity. A weighted mean of the pattern’s spectral power gives a useful numeric summary of its overall complexity, called its algebraic complexity. The basic theory of algebraic decomposition is extended in several ways, including algebraic accounts of the typicality of individual objects within concepts, and estimation of the power series from noisy data. Finally some relations between these algebraic quantities and empirical data are discussed. r 2006 Elsevier Inc. All rights reserved.